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Abstract 

Variational  formulations  have  been  proposed  for  a  number  of  tasks  in  early  vision.  Discrete 
versions  of  these  problems  are  closely  related  to  Meu^kov  random  field  models  and  are  typically  used 
in  implementing  such  methods.  In  particular,  discrete  and  continuous  versions  for  the  problem 
of  image  segmentation  have  received  considerable  attention  from  both  theoretical  eind  algorithmic 
perspectives. 

It  has  been  previously  pointed  out  that  the  usual  discrete  version  of  the  segmentation  problem 
does  not  properly  approximate  the  continuous  formulation  in  the  sense  that  the  discrete  solutions 
may  not  converge  to  a  solution  of  the  continuous  problem  as  the  lattice  spacing  tends  to  zero.  One 
method  for  modifying  the  discrete  formulations  to  ensure  such  convergence  has  been  previously 
discussed.  Here  we  consider  two  other  partially  discrete  formulations  which  also  satisfy  desirable 
convergence  properties  in  the  continumn  limit,  and  we  discuss  some  general  ideas  about  digitized 
versions  of  the  variational  formulation  of  the  segmentation  problem. 


*This  work  was  supported  by  the  U.S.  Army  Research  Office  under  Contract  DAAL03-86-K-0171,  the  Air 
Force  Office  of  Scientific  Research  under  grant  AFOSR  89-0276  and  by  the  Department  of  the  Navy  under  Air 
Force  Contract  F19628-90-C-0002. 
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1  Introduction 


Recently,  many  porblems  in  early  vision  have  been  formulated  using  variational  approaches  (e.g., 
see  [18]).  These  variational  methods  are  appealing  for  a  number  of  reasons.  The  terms  in  the  cost 
function  are  intidtively  plausible  and  correspond  in  a  natural  way  to  constraints  generally  expected 
to  be  present  in  the  environment  (for  example  certain  invariants  and/or  smoothness).  Also,  these 
methods  provide  a  tmifying  approach  to  the  wide  variety  of  early  vision  tasks,  and  in  fact  suggest 
ways  in  which  various  early  vision  modriles  might  be  fused.  Finally,  it  is  possible  to  analyze  certain 
problems  (such  as  for  segmentation)  to  obtain  general  properties  of  the  solutions  provided  by  these 
variational  techniques. 

In  practice,  these  variational  methods  are  generally  implemented  using  ftnite-difference-like  ap¬ 
proximations  with  discrete  versions  of  the  cost  function  defined  on  digitized  domains.  Interestingly, 
the  resulting  discrete  problems  are  closely  related  to  Markov  rsindom  field  (MRF)  models,  which 
are  conceptually  and  computationally  appealing.  In  fact,  for  many  problems  the  MRF  formula¬ 
tion  is  the  starting  point  from  which  a  variational  principle  on  a  continuous  domain  is  deduced  if 
necessary. 

A  continuous  formulation  is  useful  for  a  munber  of  reasons.  For  example,  it  may  be  easier 
to  impose  or  identify  certain  constraints  such  as  invariance  tmder  arbitrary  rotations  and  transla¬ 
tions.  Also,  analytical  techniques  can  be  more  readily  applied  to  derive  properties  of  the  continuous 
formulation.  However,  since  analytic  solutions  are  not  available,  the  problem  must  eventually  be 
digitized  to  obtain  numerical  solutions.  The  discrete  problem  has  the  advantages  of  being  more 
directly  amenable  to  computer  implementations,  particularly  with  parallel  algorithms  or  hardware. 
Also,  as  mentioned  above,  the  probabilistic  interpretation  in  terms  of  MRF’s  is  conceptually  ap¬ 
pealing. 

A  natural  question  is  whether  these  discrete  formulations  are  in  fact  approximations  of  the 
continuous  formulations  in  the  sense  that  solutions  to  the  discrete  problems  are  close  to  solutions 
of  the  continuous  problem  as  the  lattice  spacing  tends  to  zero.  This  question  is  important  if 
one  wants  to  guarantee  that  the  advantages  of  the  continuous  formtilations  are  retained,  at  least 
approximately,  by  solving  the  discrete  problem.  In  fact,  the  main  criteria  for  considering  a  discrete 
formulation  to  be  an  “approximation”  to  a  continuous  problem  should  not  be  whether  the  cost 
functions  are  approximations  of  one  another  in  the  usual  sense,  but  rather  whether  the  solutions 
provided  by  the  two  problems  (i.e.,  the  minimizers  of  the  cost  ftmctions)  are  approximations  of  one 
another. 

Here  we  consider  these  questions  of  suitable  discrete  approximations  for  a  particular  formulation 
for  image  restoration  and  segmentation.  Vmious  discrete  formtilations  in  terms  of  MRF’s  were 
studied  by  Geman  and  Geman  [12],  Marroquin  [17]  and  others.  A  variational  approach  to  the 
problem  was  proposed  by  Mimiford  and  Shah  in  [22]  (see  also  Blake  and  Zisserman  [4,  5]).  Below, 
we  first  describe  the  variational  method  for  the  problem  of  reconstructing  and  segmenting  an  image 
degraded  by  noise,  and  then  the  usual  discrete  approximation  proposed  for  this  problem. 

The  variational  method  involves  minimizing  a  cost  functional  over  a  space  of  boundaries  with 
suitably  smooth  functions  within  the  boimdaries.  Specifically,  if  g  represents  the  observed  image 
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defined  on  fl  C  R^,  then  a  reconstructed  image  /  and  its  associated  edges  F  are  found  by  minimizing 

E(/,r)  =  ci  ll^{f-g)^dxdy  +  C2  J \\  V  f  \\^  dx  dy  +  c^LiT)  (1) 

where  ci,C2,  C3  are  constants  and  F(r)  denotes  the  length  of  F.  An  interesting  special  case  of  this 
problem  is  obtained  if  /  is  restricted  to  be  constant  witliin  connected  components  of  n\F.  In  this 
case,  the  optimal  value  of  /  on  a  connected  component  of  fl\F  is  simply  the  mean  of  g  over  the 
coimected  component.  Hence,  the  solution  depends  only  on  F  and  is  obtained  by  minimizing 

E{T)  =  {9-  9if  dx  dy  +  C3i(F)  (2) 

i=l  •’  dOi 

where  Hi, . . . ,  Hfc  are  the  connected  components  of  n\F,  and  g^  is  the  mean  of  g  over  Hj. 

In  the  usual  discrete  versions  of  this  problem  [5,  22]  (see  also  the  MRF  formtilations  of  [12, 
17]),  the  original  image  g  is  defined  on  a  subset  of  the  lattice  with  lattice  spacing  The 
reconstructed  image  /  is  defined  on  the  same  lattice,  while  the  boundary  F  consists  of  a  subset  of 
line  segments  joining  neighboring  points  of  the  dual  lattice.  For  the  discrete  problem,  /  and  F  are 
found  by  minimizing 

Eif,T)  =  c^Y;^^{fi-gif+C2  ifi-fi'^  +  czLiV)  (3) 

ten  i.t'en 

adjacent 

7i'nr=0 

As  discussed  in  [14,  15],  it  seems  that  the  discrete  problem  given  above  does  not  properly  ap¬ 
proximate  the  continuous  problem  in  the  sense  that  solutions  to  (3)  may  not  necessarily  converge 
to  a  solution  of  (1)  as  n  — >  00.  An  alternate  discrete  formtilation  was  proposed  in  [14,  15]  which 
involved  modifications  to  both  the  cost  functional  and  the  dicretization  procedure.  The  use  of 
Minkowski  content  as  the  penalty  term  for  the  boundaries  was  considered,  and  the  discrete  boimd- 
aries  were  taken  to  be  a  union  of  closed  lattice  squares  rather  than  a  imion  of  line  segments.  It 
was  shown  that,  imder  certain  conditions,  solutions  to  tliis  discrete  formulation  converge  in  the 
continumn  limit  to  solutions  of  the  continuous  problem. 

In  tills  paper  we  discuss  some  other  possibilities  for  discrete  formulations  and  prove  similar 
convergence  results  in  these  cases.  As  in  [14,  15],  for  the  discretization  procedure  we  consider  only 
digitizing  the  boundary.  The  observed  and  reconstructed  images  are  still  defined  on  continuous 
domains.  The  alternate  formulations  are  discussed  in  Sections  3  and  4,  after  some  preliminary 
definitions  and  results  are  given  in  Section  2. 


2  Background  and  Previous  Work 

In  this  section  we  introduce  a  variety  of  notions  useful  in  dealing  with  the  ‘boundaries’  or  ‘edges’ 
of  an  image.  The  ‘image’  is  usually  a  real  valued  function  defined  on  a  bounded  open  set  H  c  R^, 
although  some  of  the  results  consider  the  more  general  case  of  H  C  R”.  A  boundary  generally 
refers  to  a  closed  subset  of  H.  However,  sometimes  the  boundary  may  be  restricted  to  have  certain 
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additional  properties  such  as  having  a  finite  number  of  connected  components.  A  topology  on  the 
space  of  boimdaries  is  required  for  the  notion  of  convergence,  and  a  measure  of  the  ‘cost’  of  a 
boundary  is  required  for  the  variational  problem. 

For  A  C  R",  the  6 -neighborhood  of  A  will  be  denoted  by  A^^^  and  is  defined  as 

A^^^  =  {*  G  R"  :  inf  \x  —  y\  < 

y£A 

The  notion  of  distance  between  boundaries  which  we  wUl  use  is  the  Hausdorff  metric  diy(-,  •)  defined 
as 

dff{Ai,  A2)  =  inf{/3  :  Ai  C  and  A2  C 

It  is  elementary  to  show  that  is  in  fact  a  metric  on  the  space  of  all  non-empty  compact 

subsets  of  R”.  An  important  and  well  known  property  of  this  metric  is  that  it  induces  a  topology 
wliich  makes  the  space  of  boundaries  compact. 

For  the  ‘cost’  of  a  boundary,  the  usual  notion  of  length  cannot  be  applied  to  highly  irregular 
boundaries.  Hence  a  measure  on  the  space  of  boundaries  which  generalizes  the  usual  notion  of 
length  is  desired.  A  variety  of  such  meastnes  for  subsets  of  R"  have  been  investigated,  (e.g.,  see 
[10]).  Perhaps  the  most  widely  used  and  studied  are  Hausdorff  measures  [9,  10,  26]. 

For  a  non-empty  subset  A  of  R”,  the  diameter  of  A  is  defined  by  |A]  =  sup{|a:  —  y]  :  a:,y  6  A}. 
Let 

n\r 

(jj  —  - lAl - 

*  r(f  +  i) 

where  r(-)  is  the  usual  Ganmia  function.  For  integer  values  of  s,  w,  is  the  volmne  of  the  unit  ball 
in  R*.  For  s  >  0  and  ^  >  0  define 

00  00 

ni{A)  =  inf{X;  :  A  C  U  <  S} 

i=:l  t=i 

The  Hausdorff  s-dimensional  measure  of  A  is  then  given  by 

H"{A)  =  limW|(A)  =  sup7f|(A) 

^-*0  s>o 

Note  that  the  factor  2“*a?,  in  the  definition  of  Hg{-)  is  included  for  proper  normalization.  With 
this  definition,  for  integer  values  of  s,  Hausdorff  measure  gives  the  desired  value  on  sets  where  the 
usual  notions  of  length,  area,  and  volume  apply. 

Many  properties  of  Hausdorff  measure  can  be  found  in  [9,  10,  26].  The  following  definitions  are 
required  to  state  several  useful  properties.  A  curve  F  C  R”  is  the  image  of  a  continuous  injection 
Ip  :  [0, 1]  — >  R”.  The  length  of  a  curve  F  is  defined  as 

m 

L(F)  =  sup{^  \xp{ti)  -  :  0  =  to  <h  <  ■■•  <tm  =  l} 

i=l 

and  F  is  said  to  be  rectifiable  if  L(F)  <  00.  Finally,  a  compact  coimected  set  is  called  a  continuum. 
Some  useful  properties  of  Hausdorff  measure  are  that  the  1-dimensioneil  Hausdorff  measure  of  a 
curve  is  equivalent  to  the  length  of  the  curve,  and  that  If  ^-measure  is  lower- semicontinuous  on  the 
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set  of  connected  boundaries  with  respect  to  the  HausdorfF  metric  —  i.e.,  if  {Fn}  is  a  sequence  of 
continuua  in  R"  that  converges  (in  HausdorfF  metric)  to  a  compact  set  T,  then  F  is  a  continuum 
and  H^T)  <  Uminf„^oo  7fi(F„). 

Richardson  [24,  25]  extended  the  lower  semicontinuity  result  to  a  cost  term  for  boundaries  which 
depends  on  the  nmnber  of  connected  components.  Specifically,  define  i/(F)  =  W^(F)  +  i^(#(r)) 
where  #(F)  denotes  the  nmnber  of  connected  components  of  F,  and  F  is  any  non- decreasing 
function  such  that  limn_too  =  oo.  Then  #(•)  and  !/(■)  are  lower-semicontinuous  on  the  space 
of  boundaries  with  respect  to  the  HausdorfF  metric.  This  result  was  used  in  [24]  to  prove  an 
existence  theorem  for  the  variational  problems  of  interest.  The  essential  properties  required  are 
the  compactness  of  the  space  of  boimdaries  and  the  lower-semicontinuity  of  the  cost  functional. 
However,  with  the  discrete  approximations  suggested  in  [5,  22]  and  elsewhere,  the  cost  for  the 
boundaries  can  be  strictly  lower-semicontinuous  with  the  topology  induced  by  the  HausdorfF  metric. 
It  is  for  this  reason  that  discrete  solutions  may  fail  to  converge  in  the  continumn  limit  to  a  solution 
of  the  continuous  problem. 

It  is  possible  to  resolve  this  problem  by  modifying  the  cost  functional  and/or  the  discretization 
process.  In  [14,  15]  an  alternate  notion  for  the  cost  of  the  boundaries  and  a  modified  discretization 
process  were  considered.  Specifically,  Minkowski  content  [10]  was  used  as  the  cost  term  for  the 
boundaries,  and  the  discrete  boundaries  were  represented  by  a  union  of  closed  lattice  squares  (as 
described  in  the  next  section)  rather  than  the  union  of  line  segments  separating  lattice  sites  as  is 
usually  done.  Several  properties  of  Minkowski  content  were  shown  in  [14,  15]  which  were  used  to 
show  convergence  properties  of  partially  discrete  versions  of  (1).  Other  approaches  to  digitizing  the 
problem  have  been  considered  and  implemented  (e.g.,  in  [25,  11]),  although  proofs  of  convergence 
of  these  methods  are  lacking  (see  Section  5  for  more  discussion).  In  the  next  two  sections  we 
consider  two  new  methods  for  digitizing  (1)  for  which  we  prove  convergence  results.  Although,  the 
methods  are  difficult  to  implement,  the  proofs  of  convergence  are  substsintially  simpler  than  that 
using  Minkowski  content.  Furthermore,  consideration  of  several  alternative  discretization  methods 
suggests  some  general  properties  of  discrete  versions  of  the  continuous  problem  as  discussed  in 
Section  5. 


3  Alternate  Cost  for  Discrete  Boundaries 

In  this  section  we  propose  a  discrete  version  of  (1)  which  possesses  the  desirable  convergence 
property  in  the  continuum  limit.  As  in  the  discretized  version  using  Minkowski  content  [14,  15], 
the  discrete  boundaries  consist  of  unions  of  discrete  closed  lattice  squares.  However,  we  assign  a 
different  cost  (measure  of  length)  to  the  discrete  boundaries.  It  is  shown  that  this  alternate  measure 
of  length  satisfies  a  convergence  property  from  which  convergence  of  solutions  to  the  variational 
problem  follows. 

For  integers  t,  j  let 


Sij  =  {(*1,3:2)  €  R^  :  i  <  *1  <  i  4-  l,i  <  *2  <  i  +  1} 

Sij  is  the  closed  unit  square  whose  lower  left  corner  is  at  Then  ^Sij  is  a  square  with  side 

i  whose  lower  left  corner  is  at  {ifn,jfn).  For  the  discrete  problem  with  lattice  spacing  F,  the 
boundaries  consist  of  a  miion  of  the  ^Sij  —  i.e.,  a  union  of  closed  lattice  squares  of  The 
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discrete  version  with  spacing  ^  of  a  boundary  F  C  is  given  by  Pn(r)  where  Pn(')  maps  F  to  the 
subset  of  R^  consisting  of  the  union  of  the  ^Sij  that  F  intersects: 

P«(r)  =  U  % 

l5<,nr^0 

We  can  now  define  an  alternate  cost  for  discrete  botmdaries  (or  rather  discrete  approximations 
to  length  for  any  boundary).  Let  F  C  R^.  First,  suppose  that  Pn(F)  (the  discrete  version  on  the 
lattice  with  spacing  1/n)  is  connected.  In  this  case,  define  £n(F)  by 

£„(F)  =  inf{7f^(A) :  A  connected  and  Pn(A)  =  F„(F)} 

In  the  general  case,  define  £n(F)  by 

£„(r)  =  5;£„(Gi) 

i 

where  the  Gi  are  the  connected  components  of  P„(F). 

The  discrete  measure  of  length  assigned  to  a  connected  discrete  boundary  P„(F)  is  the  minimum 
length  of  a  connected  curve  that  gives  rise  to  the  same  discrete  boxmdary.  Related  measures  of  length 
for  discrete  boundaries  have  been  studied  in  [13,  20,  21].  Note  that  for  bounded  F,  the  inf  could 
actually  be  replaced  by  min  since  the  infimumis  achieved.  This  follows  from  the  compactness  result 
of  compact  sets  under  the  HausdorfF  metric  and  the  lower  semicontinuity  of  Hausdorff  measure. 
The  sTun  in  the  extension  to  arbitrary  botmdaries  is  well  defined  since  any  discrete  boimdary  has 
at  most  a  countable  nmnber  of  coimected  components.  In  fact,  a  bounded  discrete  botmdary  has 
a  finite  number  of  coimected  components.  Two  boundaries  that  give  rise  to  the  same  discrete 
boundary  have  the  same  discrete  measure  of  length. 

Using  this  discrete  measure  of  length,  we  can  define  a  partially  discrete  version  of  the  original 
variational  problem  (1)  with  lattice  spacing  K  The  boundaries  are  reconstructed  only  to  within 
their  L  discrete  versions,  but  the  observed  and  reconstructed  images  are  still  defined  on  continuous 
domains.  Specifically,  given  an  observed  image  g  E  L°°(n)  we  reconstruct  a  discrete  boundary 
P„(F)  and  an  image  /  on  0  \  Pn(F)  by  minimizing 

E^f,  P„(F))  =  Cl  //  if-  gf  dxdy  +  C2  11  ||  V/  ||2  dx  dy  +  C3£n(Pn(F))  (4) 

J  J(i\Pn(.r)  J  JU\Pn{r) 

We  now  discuss  some  properties  of  the  discrete  measure  of  length  and  the  associated  discrete 
variational  problem. 

Theorem  1  For  every  fixed  n  <  oo,  a  minimizing  solution  (/*,P„(F*))  for  En{-,-)  exists. 

Proof:  For  a  fixed  boimdary,  the  minimization  of  is  a  standard  variational  problem  for  which 
a  solution  exists.  Since  fl  is  assmned  to  be  boimded,  there  are  only  a  finite  number  of  distinct 
discrete  boundaries  Pn(F),  and  so  the  result  follows.  | 


A  common  way  to  prove  existence  to  variational  problems  is  to  show  a  compactness  property  and 
lower-semicontinuity  of  the  cost  function.  Due  to  the  finite  number  of  possible  discrete  boundaries. 
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compactness  and  lower-semicontinuity  over  the  set  of  discrete  boundaries  are  immediate.  However, 
£„(•)  is  not  lower-semicontinuous  over  all  F.  For  a  simple  example,  take  n  =  1,  and  let  Ffc  be  the 
straight  line  joining  (|,|)  and  (2  —  |)|).  Then  PiCFfc)  =  5oo  U  5io  so  that  £i(rfe)  =  0.  Also, 
the  F/e  converge  to  the  straight  line  F  joining  (|,  |)  and  (2,  |),  so  that  Pi(F)  =  5oo  U  5io  U  820- 
Therefore,  £i{F)  =  1  >  liminffc_«oo  Pi(Ffe)  =  0. 

The  following  result  shows  that  for  a  large  class  of  boundaries  the  discrete  measures  of  length 
converge  to  the  usual  notion  of  length  as  n  ^  00. 

Theorem  2  If  T  is  a  compact  set  with  a  finite  number  of  connected  components  then 
liimx^oo  £n(r)  =  wi(r). 

Proof;  If  Gi, . .  -iGm  are  the  connected  components  of  F,  then  for  sufficiently  large  n  the  P„(Gj) 
are  disjoint  for  i  =  1, . . . ,  7n.  In  that  case,  £n(F)  =  ^n{Gi).  Also,  7f^(F)  = 

The  result  wiU  foUow  if  it  can  be  shown  for  each  coimected  component.  Therefore,  suppose  that  F 
is  coimected. 

By  the  definition  of  £n(F),  we  have  £n(F)  <  7f^(F)  so  that  limsup„_oo  -^nCF)  <  7f^(F).  On 
the  other  hand,  for  each  n,  there  exists  a  compact  and  connected  A„  with  P„(A„)  =  Pn(r) 
and  ■h;^(A„)  =  £n(r).  Since  P„(A„)  =  Pn(F),  A„  — >  F  in  Hausdorff  metric  as  n  — >  00.  By 
the  lower- semicontinuity  of  V}  (see  [9],  Theorem  3.18)  we  have  Ti}{T)  <  liminfn-.oo 'H^(A„)  = 
liminf„_,oo  £n{F).  Thus,  limn-»oo  £n(F)  exists  and  equals  'K^(F).  | 


The  following  theorem  shows  that  the  discrete  measures  of  length  converge  in  another  useful 
way  to  Hausdorff  measure. 

Theorem  3  £ti(')  is  T -convergent  to  on  the  space  of  compact  subsets  o/R^  with  a  bounded 

number  of  connected  components  and  with  the  topology  induced  by  the  Hausdorff  metric.  Le.,  let 
F  C  R^  6e  compact  with  #(F)  <  M  <  00.  Then  the  following  two  conditions  hold: 

(i)  For  every  sequence  of  compact  sets  Fn  C  R^  with  F„  — >  F  in  Hausdorff  metric  and  #(F„)  < 

M  Vn  we  have 

n\T)  <  liminf£n(rn) 

n — ►oo 

(ii)  There  exists  a  sequence  of  compact  sets  F„  C  R^  with  F„  — F  in  Hausdorff  metric  and 

#(F„)  <  M  'in  such  that 

H^F)  >limsup£„(F„) 


Proof:  To  show  (i),  let  F„  be  any  sequence  of  compact  sets  with  #(F„)  <  M  and  F„  ^  F  in 
Hausdorff  metric.  Then  #(P„(F„))  <  #{F„)  <  M.  Therefore,  for  each  n  there  exists  A„  with 
#(A„)  =  #(P„(F„))  <  M,  P„(A„)  =  P„(F„),  and  V}{K)  =  £n(F„).  Since  F„  ^  F,  we  have 
Pn{rn)  r  and  also  since  P„(A„)  =  P„(F„)  we  have  A„  — >  F.  Hence,  by  the  lower- semicontinuity 
of  Hausdorff  measure  for  a  bounded  number  of  connected  components  (as  mentioned  in  Section  2 
—  see  [24,  25]),  we  get  ?f^(F)  <  liminfn->oo  W^(A„)  =  liminf„_*oo  £n(rn)- 
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To  show  (ii),  simply  take  r„  =  F  for  all  n.  Then  by  the  definition  of  £n(r),  we  have  7f^(r)  > 
Cn(G)  =  £„{r„)  for  all  n  and  so  H^(r)  >  limsup„_^  £n(r„).  | 


The  F-convergence  property  shown  above  is  sufficient  to  show  convergence  of  solutions  to  the 
discrete  problem  (4)  as  the  lattice  spacing  ^  goes  to  zero  if  the  number  of  connected  components 
of  the  admissible  boundaries  is  uniformly  bounded.  I.e.,  following  [24],  we  let  the  cost  term  for  the 
boimdaries  be 

Mr)  =  £n{r)  +  F(#(F)) 

where  F{k)  =  Q  fox  k  <  M  <  oo  and  F{k)  =  oo  for  k  >  M.  Let  denote  the  cost  functional 
with  the  above  botmdary  term,  and  let  denote  the  cost  ftmctional  whose  boundary  term  is 

i/(F)  =  ?fHr)  +  F(#(F)) 

For  these  variational  problems,  we  have  the  following  convergence  result,  which  essentially  follows 
from  the  previous  result  on  F-convergence. 

Theorem  4  Let  (/^,F*)  denote  a  minimizing  pair  for  Eff.  Then  there  exists  a  subsequence  (still 
denoted  (/^,F*)j  and  a  pair  (/,  F)  such  that  (/*,F*)  — >  (/,F)  and  (/,  F)  minimizes  E^ . 

Proof:  The  existence  of  a  pair  (/,  F)  with  (/*,F*)  — >  (/,  F)  follows  from  Lemma  3  of  [24].  The 
fact  that  (/,  F)  minimizes  E^  follows  from  the  F-convergence  of  £„  to  Ti}  (ahown  above)  which 
implies  the  F-convergence  of  E^  to  E^.  | 


4  Segmentation  with  Piecewise  Linear  Boundaries 

In  this  section  we  formulate  a  modified  version  of  the  original  variational  problem  (1)  which  elim¬ 
inates  the  problems  associated  with  computing  the  length  of  irregular  curves.  The  modification 
consists  of  requiring  the  botmdaries  F  to  consist  of  a  union  of  a  botmded  nmnber  of  straight  line 
segments.  We  propose  a  partially  discrete  version  of  the  modified  problem  and  show  a  convergence 
result  for  the  solutions  to  the  discrete  problems. 

Let  =  «S^(n)  be  the  set  of  all  compact  subsets  of  fl  that  are  the  miion  of  M  <  oo  or  fewer 
connected  line  segments  contained  in  fi.  Consider  the  problem  of  minimizing  (1)  subject  to  the 
constraint  that  F  G  S^.  Alternatively,  we  can  consider  this  problem  as  one  of  minimizing  the  cost 
function  EgM  whose  boundary  term  is  7f^(F)  for  F  €  and  infinite  otherwise. 

Theorem  6  For  every  fixed  M  <  oo  and  Cl  a  bounded  region  ofH^,  5^(0)  is  compact  with  respect 
to  the  Hausdorff  metric.  I.e.,  for  every  infinte  sequence  F„  G  there  is  a  subsequence,  still 

denoted  F„,  and  F  G  <S^(0)  such  that  Fn  ^  F  in  Hausdorff  metric. 
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Proof;  First,  each  r„  is  a  compact  subset  of  As  mentioned  in  Section  2,  the  set  of  compact 
subsets  of  fl  is  compact  with  respect  to  the  Hausdorff  metric  (e.g.,  see  [9],  Theorem  3.16).  Hence, 
there  is  a  subsequence,  which  we  still  denote  r„,  and  a  compact  set  F  C  such  that  Fn  — >  F  in 
Hausdorff  metric.  We  need  only  show  that  F  G  i.e.,  that  F  is  the  union  of  at  most  M  line 

segments. 

Note  that  by  a  line  segement  we  include  the  possibility  of  a  single  point  or  the  empty  set.  It 
is  straightforward  to  show  the  result  for  M  =  1.  I.e.,  given  a  sequence  of  single  line  segments  any 
convergent  subequence  converges  to  a  single  line  segment.  Now,  each  F„  is  the  union  of  at  most 
M  line  segments  so  that  F„  =  where  each  An,i  is  a  line  segment  (possibly  a  point  or  the 

empty  set). 

We  can  extract  a  subsequence  (still  indexed  by  n)  so  that  ^  Ai  where  Ai  is  a  line  segment 
(again,  possibly  a  point  or  the  empty  set).  Similarly,  we  can  extract  sub-subsequences  M  —  1  more 
times  (still  indexed  by  n)  so  that  for  each  i  =  1, . . . ,  M,  An,i  —*  Ai  where  each  Ai  is  a  line  segment. 

Therefore,  for  the  final  sub-subsequence,  F„  =  and  for  each  i  =  l,...,m  we  have 

An.i  —*  Ai  where  Ai  is  a  line  segment.  Hence,  Fn  — >  F  =  and  so  F  G  <5^(0).  | 


Theorem  6  A  minimizing  solution  (/*,F*)  for  E^m{-,-)  exists. 

Proof:  This  follows  from  the  compactness  of  shown  above  emd  the  lower- semicontinuity  of  the 
cost  function  (e.g.,  see  [24,  25]).  | 


Now  we  consider  partially  discrete  versions  of  the  problem  E^m.  As  before,  oidy  the  boimdary 
is  discretized.  For  this  problem  we  consider  a  different  form  of  discrete  representation  for  the 
boundaries,  taking  advantage  of  the  fact  that  the  boundaries  are  piecewise  linear.  Specifically, 
we  consider  the  set  of  lattice  points  in  1/nZ^,  and  require  the  endpoints  of  each  line  segment  in 
the  boundary  to  lie  on  these  lattice  points.  Let  5^’”  =  denote  the  collection  of  aU  sets 

consisting  of  M  straight  line  segments  whose  endpoints  lie  in  nF„(n).  To  obtain  a  convergence 

result  we  will  consider  the  boimdary  to  be  a  dilation  of  the  linear  segments  where  the  amount  of 
dilation  is  related  to  the  lattice  spacing.  For  the  discrete  problem  with  lattice  spacing  1/n,  the  cost 
function  is  minimized  over  boundaries  of  the  form  F^*")  where  Sn  >  V^jn  with  >  0  as  n  — >  oo 
and  F  G  <S'^’”(n).  The  reconstructed  image  is  then  defined  only  on  O  \  However,  the  cost  of 

the  dilated  boundary  F^*"^  is  taken  to  be  simply  the  the  total  length  of  the  straight  line  segments 
comprising  F.  Let  E^M.n  denote  the  discrete  problem  with  lattice  spacing  1/n.  For  these  discrete 
problems,  we  have  the  following  convergence  result. 

Theorem  7  Let  (/*,F*)  denote  a  minimizing  pair  for  EgM.n.  Then  there  exists  a  subsequence 
(still  denoted  (/^,F*)/  and  a  pair  (/,  F)  such  that  (/*,F*)  — >  (/,  F)  and  (/,  F)  minimizes  EgM. 

Proof;  The  existence  of  a  pair  (/,  F)  with  (/,t,F*)  ^  (/»F)  follows  from  the  compactness  of 
and  Lemma  3  of  [24].  We  only  need  to  show  that  (/,  F)  minimizes  Egia. 
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Let  (/*,  r*)  minimize  Esm-  For  each  n,  let  A„  be  obtained  from  F*  by  taking  the  best  approxi¬ 
mation  to  r*  using  line  segments  whose  endpoints  lie  in  n  P„(n),  i.e,  the  best  boundary  which 
can  be  used  in  the  discrete  problem  with  lattice  spacing  This  will  be  obtained  by  taking  the  line 
segments  whose  endpoints  lie  on  the  lattice  sites  closest  to  the  endpoints  of  F*.  Note  that  certainly 
D  F*  since  6n  >  V^/n  and  with  lattice  spacing  F  there  is  always  a  lattice  point  within  \/2/n 
of  any  point  in  fi.  Let  /i„  be  the  restriction  of  /*  to  Then,  we  have 

EsM{f,T)  <  ]im\TdEsM,n{f*,T^)  <]im.ii3£EsM,n{hn,An) 

H — ^oo  n — >00 

=  lim  EsM,n{hn,A„)  =  Esn.{r,n 

n— »oo  '  /  V  \ 


where  the  first  inequality  uses  the  lower-semicontinuity  of  E^m  (which  follows  from  results  from 
[24]),  the  second  inequality  follows  from  the  optimality  of  (/•,F*),  and  the  equalities  follow  from 
the  continuity  of  the  first  two  terms  of  E^m  with  repect  to  dilations  of  the  boimdary  (see  [24]). 
Therefore,  E^m(/,  F)  =  E^M{f*,T*)  so  that  (/,F)  minimizes  E^m.  | 


5  Discussion  and  Summary 

We  presented  two  procedures  for  partially  discretizing  the  variationeJ  formulation  of  the  segmen¬ 
tation  problem  for  which  we  proved  convergence  results  in  the  continuum  limit.  This  problem  was 
originally  considered  in  [14, 15]  where  a  method  like  that  of  Section  3  (but  using  Minkowski  content 
for  the  length  term)  was  shown  to  have  similar  convergence  properties.  There  are  several  reasons 
for  considering  other  methods  such  as  those  presented  here.  First,  the  proofs  of  the  convergence  re¬ 
sults  for  the  techniques  presented  here  are  much  simpler  (using  some  powerful  results  on  Hausdorff 
measure)  than  that  of  [14,  15].  Second,  certain  discretization  methods  and  cost  terms  may  be  easier 
to  implement  and  computationally  more  advantageous  than  others.  Finally,  considering  several  al¬ 
ternative  approaches  might  suggest  certain  general  properties  that  are  shared  by  all  discretizations 
of  the  originaJ  problem. 

Regarding  the  second  and  third  points  we  have  some  specific  ideas  in  mind.  Both  approaches 
presented  here  as  weU  as  the  method  using  Minkowski  content  are  computationally  unattractive 
compared  to  the  standard  discretization.  For  example,  in  the  standard  discretization  the  cost  term 
associated  with  a  discrete  boundary  is  obtained  by  simply  taking  the  total  length  of  the  segments 
in  the  discrete  boundary.  On  the  other  h€ind,  with  the  digitizations  proposed,  the  computation  of 
the  cost  term  of  Section  3  or  of  Minkowski  content  is  much  more  involved.  In  particular,  the  MRF 
corresponding  to  the  standard  discretization  is  very  simple,  requiring  no  interactions  between  the 
boundary  sites,  while  implementing  the  discretizations  proposed  would  require  large  neighborhood 
structures  (growing  imboundedly  as  the  lattice  spacing  tends  to  zero)  and  complex  potentials. 

The  distinction  can  also  be  formalized  along  the  following  lines.  Consider  a  distributed  imple¬ 
mentation  in  which  there  is  a  processor  at  each  lattice  site.  The  state  of  a  processor  is  either  zero 
or  one  depending  on  whether  or  not  the  boundary  passes  through  the  associated  lattice  square.  To 
compute  the  length  terms  of  Section  2  or  using  Minkowski  content,  each  processor  must  perform  a 
computation  depending  on  the  state  of  a  very  large  number  (tending  to  infinity)  of  other  processors 
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as  the  lattice  spacing  tends  to  zero.  On  the  other  hand,  for  the  cost  term  of  the  usual  discretization, 
the  contribution  of  a  particular  processor  to  the  total  cost  depends  on  the  state  of  the  processor  but 
is  independent  of  the  state  of  all  other  processors  (regardless  of  the  discretization  level).  Hence,  if 
implemented  in  parallel  architectures  in  the  natural  way,  the  two  methods  that  possess  the  proper 
convergence  properties  require  computations  that  are  in  some  sense  nonlocal  as  the  lattice  spacing 
tends  to  zero,  while  the  usual  discretization  results  in  a  local  computation  (independent  of  the 
discretization  level)  but  fails  to  have  the  right  convergence  properties.  Note  that  for  the  method 
using  piecewise  linear  approximations,  if  implemented  in  the  natural  way,  the  computation  can  be 
done  locally  but  each  processor  requires  an  unbounded  number  of  states  in  the  continuum  limit  (to 
indicate  whether  an  endpoint  of  the  line  segment  is  present  at  that  processor  and,  if  so,  at  which 
processor  the  other  endpoint  lies). 

A  natural  question  is  whether  the  computational  difficulties  discussed  above  can  be  circmnvented 
by  a  clever  discrete  approximation.  We  have  obtained  some  results  suggesting  that  for  rectcuigular 
lattices  the  difficulties  are  not  merely  due  to  a  poor  choice  of  discrete  approximations,  but  are 
inherent  difficulties  associated  with  any  discrete  approximation  to  measures  of  length.  This  result 
probably  holds  for  many  other  regular  lattices  as  well  (e.g.,  hexagonal).  However,  interestingly  the 
problems  with  nonlocal  computation  can  be  avoided  for  certain  random  tesselations  (and  probably 
for  appropriate  deterministic  ones  as  well). 

Also,  as  alluded  to  above,  the  nonlocal  computations  can  likely  be  avoided  if  the  processors  are 
allowed  to  have  infinitely  many  states.  For  example,  this  could  correspond  to  associating  a  direction 
(or  local  tangent)  to  each  boundary  element  in  addition  to  just  it  presence  or  absence.  Hence,  in  the 
MRF  formulations  this  might  correspond  to  coupled  intensity  and  botmdary  fields  both  of  which  are 
real  valued.  A  somewhat  different  approach  to  having  real  valued  boundary  elements  is  suggested 
by  an  important  restdt  of  Ambrosio  [1,  2].  He  obtained  an  interesting  F-convergent  approximation 
to  the  original  variationed  problem.  Specifically,  he  showed  that  the  functional 

=  c.  f/jf  -  s)*  +  c,  I £(1  -  II  Vf  II*  +C3(  f/jl-  .;*)'■  II  V, 

F-converges  to  (1)  as  h  —>■  oo,  so  that  minimizers  of  E^{f,v)  converge  to  a  minimizer  of  jF(/,F) 
as  h  — >  oo.  Here  /  is  as  before  and  u  :  O  [0, 1]  plays  the  role  of  the  botmdaries.  For  finite 
h,  V  represents  a  sort  of  smoothed  version  of  F  in  the  sense  of  having  a  value  close  to  1  near  F 
and  having  a  value  of  0  away  from  F,  and  varying  continuously  in  between.  This  result  suggests 
a  natural  digitization  of  (1)  by  taking  a  finite  difference  approximation  to  E^{f,v)  as  discussed 
in  [25]  and  [11].  However,  as  far  as  we  know,  a  proof  of  convergence  for  such  finite  difference 
approximations  is  lacking  in  this  case.  We  expect  that  convergence  should  hold  as  long  as  h  — >  oo 
appropriately  as  the  lattice  spacing  1/n  -+  0,  namely  ft/n  — >  0.  Such  a  conjecture  is  natural  in  light 
of  the  results  of  [14,  15]  and  was  in  fact  stated  in  [25].  Furthermore,  convergence  issues  aside,  it 
is  not  clear  that  computational  difficulties  are  avoided  with  these  approaches.  E.g.,  in  Ambrosio’s 
approximation,  there  may  be  some  computationeil  or  numerical  problems  as  h  — >  oo.  Further  work 
needs  to  be  done  to  understand  whether  any  computational  difficulties  arise  in  this  case. 
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